Existing methods for large-scale point cloud semantic segmentation require expensive, tedious and error-prone manual point-wise annotations. Intuitively, weakly supervised training is a direct solution to reduce the cost of labeling. However, for weakly supervised large-scale point cloud semantic segmentation, too few annotations will inevitably lead to ineffective learning of network. We propose an effective weakly supervised method containing two components to solve the above problem. Firstly, we construct a pretext task, \textit{i.e.,} point cloud colorization, with a self-supervised learning to transfer the learned prior knowledge from a large amount of unlabeled point cloud to a weakly supervised network. In this way, the representation capability of the weakly supervised network can be improved by the guidance from a heterogeneous task. Besides, to generate pseudo label for unlabeled data, a sparse label propagation mechanism is proposed with the help of generated class prototypes, which is used to measure the classification confidence of unlabeled point. Our method is evaluated on large-scale point cloud datasets with different scenarios including indoor and outdoor. The experimental results show the large gain against existing weakly supervised and comparable results to fully supervised methods\footnote{Code based on mindspore: https://github.com/dmcv-ecnu/MindSpore\_ModelZoo/tree/main/WS3\_MindSpore}.

移动机器人的指导任务不仅需要人类感知导航，还需要适当但及时交互以获得活动指令。最先进的导游模型限制了他们的社会知识的考虑，以适应用户的运动，忽略互动行为计划以满足交际需求。我们提出了一种基于Monte Carlo树搜索的多行为规划框架，以更好地帮助用户了解令人困惑的场景上下文，选择正确的路径并及时到达目的地。为了提供主动指导，我们构建了一种基于采样的人类运动概率模型，以考虑机器人和人类之间的相互关联的影响。我们在模拟和现实世界实验中验证了我们的方法以及与最先进模型的性能比较。

In this paper, we propose PanoViT, a panorama vision transformer to estimate the room layout from a single panoramic image. Compared to CNN models, our PanoViT is more proficient in learning global information from the panoramic image for the estimation of complex room layouts. Considering the difference between a perspective image and an equirectangular image, we design a novel recurrent position embedding and a patch sampling method for the processing of panoramic images. In addition to extracting global information, PanoViT also includes a frequency-domain edge enhancement module and a 3D loss to extract local geometric features in a panoramic image. Experimental results on several datasets demonstrate that our method outperforms state-of-the-art solutions in room layout prediction accuracy.

点云的几乎没有分割仍然是一项具有挑战性的任务，因为没有有效的方法将局部点云信息转换为全局表示，这阻碍了点特征的概括能力。在这项研究中，我们提出了双向特征全球化（BFG）方法，该方法利用点特征和原型向量之间的相似性测量，以双向方式将全球感知嵌入到局部点特征中。随着点对点型全球化（PO2PRG），BFG根据从密度点特征到稀疏原型的相似权重将本地点特征汇总到原型。使用原型到点全球化（PR2POG），基于从稀疏原型到密集点特征的相似性权重，全局感知嵌入到局部点特征中。每个类嵌入全局感知的类的稀疏原型汇总到基于度量学习框架的几个原型3D分割的单个原型。对S3DIS和SCANNET的广泛实验表明，BFG显着超过了最新方法。

服务机器人安全有礼貌的机器人需要坚强地跟踪周围人，尤其是对于旅游指南机器人（TGR）。但是，由于以下原因，现有的多对象跟踪（MOT）或多人跟踪（MPT）方法不适用于TGR：1。缺乏相关的大型数据集；2.缺少适用的指标来评估跟踪器。在这项工作中，我们针对TGR的视觉感知任务，并介绍TGRDB数据集，TGRDB数据集是一种新颖的大型多人跟踪数据集，其中包含大约5.6小时的带注释视频和超过450个长期轨迹。此外，我们提出了一个更适合使用数据集评估跟踪器的指标。作为我们工作的一部分，我们提出了TGRMPT，这是一种新型的MPT系统，它结合了头部肩膀和全身的信息，并实现了最先进的性能。我们已经在https://github.com/wenwenzju/tgrmpt中发布了代码和数据集。

Modern object detectors have taken the advantages of backbone networks pre-trained on large scale datasets. Except for the backbone networks, however, other components such as the detector head and the feature pyramid network (FPN) remain trained from scratch, which hinders fully tapping the potential of representation models. In this study, we propose to integrally migrate pre-trained transformer encoder-decoders (imTED) to a detector, constructing a feature extraction path which is ``fully pre-trained" so that detectors' generalization capacity is maximized. The essential differences between imTED with the baseline detector are twofold: (1) migrating the pre-trained transformer decoder to the detector head while removing the randomly initialized FPN from the feature extraction path; and (2) defining a multi-scale feature modulator (MFM) to enhance scale adaptability. Such designs not only reduce randomly initialized parameters significantly but also unify detector training with representation learning intendedly. Experiments on the MS COCO object detection dataset show that imTED consistently outperforms its counterparts by $\sim$2.4 AP. Without bells and whistles, imTED improves the state-of-the-art of few-shot object detection by up to 7.6 AP. Code is available at https://github.com/LiewFeng/imTED.

点云的语义分割通过密集预测每个点的类别来产生对场景的全面理解。由于接收场的一致性，点云的语义分割对于多受感受性场特征的表达仍然具有挑战性，这会导致对具有相似空间结构的实例的错误分类。在本文中，我们提出了一个植根于扩张图特征聚集（DGFA）的图形卷积网络DGFA-NET，该图由通过金字塔解码器计算出的多基质聚集损失（Maloss）引导。为了配置多受感受性字段特征，将建议的扩张图卷积（DGCONV）作为其基本构建块，旨在通过捕获带有各种接收区域的扩张图来汇总多尺度特征表示。通过同时考虑用不同分辨率的点集作为计算碱基的点集惩罚接收场信息，我们引入了由Maloss驱动的金字塔解码器，以了解接受田间的多样性。结合这两个方面，DGFA-NET显着提高了具有相似空间结构的实例的分割性能。 S3DIS，ShapenetPart和Toronto-3D的实验表明，DGFA-NET优于基线方法，实现了新的最新细分性能。

稀疏培训是一种自然的想法，可以加速深度神经网络的训练速度，并节省内存使用，特别是因为大型现代神经网络被显着过度参数化。然而，大多数现有方法在实践中无法实现这一目标，因为先前方法采用的基于链规则的梯度（W.R.T.结构参数）估计。至少在向后传播步骤中至少需要密集的计算。本文通过提出具有完全稀疏的前后通行证的有效稀疏训练方法来解决这个问题。我们首先在全球稀疏限制下将培训过程制定为连续最小化问题。然后，我们将优化过程分为两个步骤，对应于权重更新和结构参数更新。对于前一步，我们使用传统的链规则，这可以通过利用稀疏结构来稀疏。对于后一步，而不是使用基于链规则的梯度估计器，如现有方法中，我们提出了一个方差减少的策略梯度估计器，这只需要两个向前通过而不向后传播，从而实现完全稀疏的训练。我们证明了我们渐变估计器的差异是界定的。对现实世界数据集的广泛实验结果表明，与以前的方法相比，我们的算法在加速训练过程中更有效，速度快到速度更快。

In this paper, we propose a robust 3D detector, named Cross Modal Transformer (CMT), for end-to-end 3D multi-modal detection. Without explicit view transformation, CMT takes the image and point clouds tokens as inputs and directly outputs accurate 3D bounding boxes. The spatial alignment of multi-modal tokens is performed implicitly, by encoding the 3D points into multi-modal features. The core design of CMT is quite simple while its performance is impressive. CMT obtains 73.0% NDS on nuScenes benchmark. Moreover, CMT has a strong robustness even if the LiDAR is missing. Code will be released at https://github.com/junjie18/CMT.

Dataset distillation has emerged as a prominent technique to improve data efficiency when training machine learning models. It encapsulates the knowledge from a large dataset into a smaller synthetic dataset. A model trained on this smaller distilled dataset can attain comparable performance to a model trained on the original training dataset. However, the existing dataset distillation techniques mainly aim at achieving the best trade-off between resource usage efficiency and model utility. The security risks stemming from them have not been explored. This study performs the first backdoor attack against the models trained on the data distilled by dataset distillation models in the image domain. Concretely, we inject triggers into the synthetic data during the distillation procedure rather than during the model training stage, where all previous attacks are performed. We propose two types of backdoor attacks, namely NAIVEATTACK and DOORPING. NAIVEATTACK simply adds triggers to the raw data at the initial distillation phase, while DOORPING iteratively updates the triggers during the entire distillation procedure. We conduct extensive evaluations on multiple datasets, architectures, and dataset distillation techniques. Empirical evaluation shows that NAIVEATTACK achieves decent attack success rate (ASR) scores in some cases, while DOORPING reaches higher ASR scores (close to 1.0) in all cases. Furthermore, we conduct a comprehensive ablation study to analyze the factors that may affect the attack performance. Finally, we evaluate multiple defense mechanisms against our backdoor attacks and show that our attacks can practically circumvent these defense mechanisms.